Use of digital images has become commonplace in both business and recreational environments. This increased usage results in people increasingly seeking to insert images into documents such as word processing documents, presentations, spreadsheets or social networking applications. It is often desirable to remove the background of an image document to remove unnecessary information from the image. For example, in images that have been generated by a document scanner or camera there may be unwanted background information. A common approach is to attempt to remove the background of an image document as a whole. However, image documents are often affected by different lighting in different areas of the document. Removing the background of entire image documents often generates noise in the dark portions of the document while causing image information in the lighter areas of the document to become too light or even get lost.